Audio-Controlled Image Capturing

ABSTRACT

A method comprising: receiving a plurality of images corresponding to a time period covering the intended moment for releasing the shutter; receiving an audio signal associated with the plurality of images using audio capturing means; analyzing the received audio signal in order to determine an auditory event associated with a desired output image; and selecting at least one of the plurality of images on the basis of the analysis of the received audio signal for further processing in order to obtain the desired output image.

FIELD OF THE INVENTION

The present invention relates to imaging, and more particularly tocontrolling still image capturing according to audio information.

BACKGROUND OF THE INVENTION

In the field of image capturing technology, along with the aim ofensuring the best possible image quality, many cameras have beenprovided with a so-called multi-shot functionality. In the multi-shotfunctionality, when the user has released the shutter, the imagecapturing device is arranged to take several images in order to capturethe right moment. The user may then later on select the best one(s) ofthe multitude of images. In a more advanced solution, the user or theimage capturing device may combine several images, or all the capturedimages, into one composed output image with enhanced visual quality.

In a further embodiment of the multi-shot functionality, it is known totake multiple images with different exposure times and select thedesired one(s) as the output image, or for further processing to derivea composed output image with enhanced visual quality.

However, there still remain some disadvantages. Capturing a picture maysometimes require quick reactions in order to catch the picture the userwants to capture. Especially in case the object of the picture ismoving, the timing of the picture might easily be non-optimal, theobject may have escaped the image and the desired moment is lost.Furthermore, in the known multi-shot solutions, selecting the bestquality image or composing an enhanced output image always requirespost-processing of the captured images, which the user may experiencetroublesome and time consuming.

SUMMARY OF THE INVENTION

Now there has been invented an improved method and technical equipmentimplementing the method, by which the user of the image capturing deviceis assisted to take a picture with improved quality. Various aspects ofthe invention include a method, an apparatus and a computer program,which are characterized by what is stated in the independent claims.Various embodiments of the invention are disclosed in the dependentclaims.

According to a first aspect, a method according to the invention isbased on the idea that of receiving a plurality of images correspondingto a time period covering an intended moment; receiving an audio signalassociated with the plurality of images using audio capturing means;analyzing the received audio signal in order to determine an auditoryevent associated with a desired output image; and selecting at least oneof the plurality of images on the basis of the analysis of the receivedaudio signal for further processing in order to obtain the desiredoutput image.

According to an embodiment, the method further comprises receiving theplurality of images comprises capturing the plurality of images duringthe period covering the intended moment using an image capturing unit inresponse to a user of an image capturing device giving a command torelease a shutter at the intended moment,; and receiving the audiosignal comprises capturing an audio signal associated with the pluralityof images using audio capturing means.

According to an embodiment, the method further comprises receiving aplurality of images corresponding to a first time instant before theintended moment.

According to an embodiment, the method further comprises receiving aplurality of images corresponding to a second time instant after theintended moment.

According to an embodiment, the audio receiving means comprise one ormore microphones.

According to an embodiment, the audio receiving means comprise amicrophone array consisting of two or more microphones.

According to an embodiment, the method further comprises estimating thedirection of arrival of the audio signal in order to determine when theobject producing the audio signal is in desired part of the image.

According to an embodiment, the object producing the audio signalresiding substantially perpendicular to the microphone array is used todetermine when the object is in desired part of the image

According to an embodiment, the method further comprises estimatingenergy of the receiving audio signal in order to determine an event ofinterest within the plurality of images.

According to an embodiment, an abrupt change in the energy of thereceiving audio signal is used to determine the desired moment of theevent of interest.

According to an embodiment, the method further comprises estimating afundamental frequency of the receiving audio signal in order todetermine at least one image from the plurality of images in which theobject producing the audio signal is closest to the image capturingdevice providing the plurality of images.

According to an embodiment, the method further comprises using theestimation of the fundamental frequency to select an image in which theobject producing the audio signal is approaching the image capturingdevice providing the plurality of images.

According to an embodiment, the method further comprises obtaininginformation relating to a distance between the image capturing deviceproviding the plurality of images and the object producing the audiosignal; and adjusting the capturing of the audio signal with a delaycaused by the speed of sound on said distance.

The arrangement according to the invention provides significantadvantages. On a general level, it enables or assists a user of theimage capturing device to take a picture with improved quality byanalysing an audio signal associated with a plurality of images in orderto determine an optimal output image based on the plurality of images.More particularly, the user may capture better quality images byautomatically selecting, from the set of images, for example the imagein which the object of interest resides in the middle of the viewfinder,or a special audio event (e.g. a crash or explosion) happens, or theobject is approaching the camera (front view). The output image isautomatically selected at a moment of time, which is closest to adesired audio event.

According to a second aspect, there is provided an apparatus comprising:an image receiving unitfor receiving a plurality of images correspondingto a time period covering an intended moment; an audio receiving unitfor receiving an audio signal associated with the plurality of imagesusing audio capturing means; an analysing unit for analyzing thereceived audio signal in order to determine an auditory event associatedwith a desired output image; and a selecting unit for selecting at leastone of the plurality of images on the basis of the analysis of thereceived audio signal for further processing in order to obtain thedesired output image.

These and other aspects of the invention and the embodiments relatedthereto will become apparent in view of the detailed disclosure of theembodiments further below.

LIST OF DRAWINGS

In the following, various embodiments of the invention will be describedin more detail with reference to the appended drawings, in which

FIG. 1 shows an example of a series of multi-shot images;

FIG. 2 shows an example of a multi-microphone array arranged in an imagecapturing device;

FIG. 3 shows an example of functional units implemented according to anembodiment of the invention;

FIG. 4 shows a generic microphone array consisting of a plurality ofmicrophones in a beamformer configuration;

FIG. 5 shows an example of sound direction of arrival estimation for acaptured audio signal;

FIG. 6 shows an example of sound level estimation for the captured audiosignal; and

FIG. 7 shows an example of the fundamental frequency estimation for thecaptured audio signal.

DESCRIPTION OF EMBODIMENTS

In the following, the invention will be illustrated by referring toimage capturing devices in general, occasionally referring to cameradevices as well-known examples. It is, however, noted that the inventionis not limited to stand-alone cameras solely, but it can be implementedin any image capturing device comprising an image sensor and means forprocessing the captured image and audio information. For example, theinvention may be utilized in a mobile terminal, PDA device, a handheldcomputer or a laptop computer equipped with a camera module. Therefore,the invention must also be considered from the perspective of such anapparatus receiving images from an image capturing unit and/or audioinformation from an audio capturing unit, and the apparatus thenprocesses the received image and audio information as described in theembodiments.

The embodiments described below provide a method for controlling theimage capturing device, when it is operated in the multi-shot operationmode. In the multi-shot operation mode, the image capturing device, e.g.a camera, is arranged to take a plurality of images every time theshutter is released. The number of images taken in the multi-shotoperation mode may be a default setting of the image capturing device orit may be adjustable by the user in the device settings, and the numberof captured images may vary from two up to tens or even hundreds ofimages. The examples described in the embodiments below are based on thesetting of capturing ten images in each multi-shot series.

In the multi-shot operation mode, the image capturing device isgenerally arranged to take a plurality of images during a period, whichcovers, i.e. includes, the intended moment for releasing the shutter.Preferably, taking the series of images may begin already before theshutter button is firmly pressed down, for example already when thebutton is pressed half way, which enables to adjust the focus for thesubsequent images. The camera may be arranged to store a predefinednumber of images before and, respectively after the button was firmlypressed down. For example, in a multi-shot series of ten images firstfive images could be arranged to be captured after the button is thehalf way but before it is completely pressed down, and the remainingfive images are then captured after the button has been completelypressed down.

Regardless of the number of images taken in a multi-shot series, theimage capturing device may still comprise a pre-determined margin valuecontrolling the operation such that the images captured earlier than thepre-determined margin before the shutter is completely released could besubsequently discarded without further processing or saving them in thememory.

FIG. 1 presents an example of a series of first five pictures taken ofan object, i.e. a passing race car in this case, when the user haspressed the button. The latter five pictures are not shown in thefigure.

The same example of FIG. 1 is utilised in illustrating the embodimentsdescribed below.

According to the embodiments, the image capturing device, when operatedin the multi-shot operation mode and capturing the multi-shot series ofa plurality of images, is also arranged to capture an audio signalassociated with the images in the multi-shot series. The time segmentfor which the audio signal is captured may be a default setting of theimage capturing device or it may be adjustable by the user in the devicesettings. The audio capturing preferably starts substantially at thesame time when the image capturing starts. Thus, the image capturingdevice is preferably arranged to capture a suitable time segment, e.g.five seconds, of audio associated with the series of images before theuser has pressed the button completely down, and respectively anothersuitable time segment after the user has pressed the button completelydown.

The image capturing device may store the captured audio signal in amemory for subsequent audio analysis. Alternatively, the audio analysis,as described in detail below, may be performed immediately withoutstoring the full audio segment in the memory.

The purpose of analysis of the captured audio signal associated with theimages in the multi-shot series is to identify the moment(s) when one ormore images of the multi-shot series most probably have captured anoptimal quality image of the desired object. The analysis of thecaptured audio signal associated with the plurality of images may bebased on determining the direction of arrival of major sound componentsin order to determine when the sound source (i.e. the object) is indesired part of the image, or any discontinuation in the sound level,spectral component or fundamental frequency may be used to invoke theselection algorithm to select one or more images for further processing.

When the most promising time moment has been identified, the one or moreimages of the multi-shot series associated with said moment can beselected as the optimal quality images. An optimal quality image maythen be selected as an output image as such, or one or more images maybe selected for further processing in order to produce a better qualityoutput image.

The user is also provided with the possibility to select some otherimage as the output image than what is the optimal quality imagesuggested by the analysis. For instance, in the example of FIG. 1, theuser can select the output image be the one captured when the race caris situated in the right part of the viewfinder. In this manner, thecaptured images can contain some other parts of the scene that areimportant for the user.

The audio signal could be captured with one or more microphones eitherintegrated in the image capturing device or being functionally connectedand residing preferably in close proximity to the image capturingdevice. Especially, if the direction of arrival of major soundcomponents needs to be determined, then a microphone array consisting oftwo or more microphones, preferably mounted on the image capturingdevice could be used. An example of such multi-microphone array ispresented in FIG. 2, which shows the tracked object 200, an image ofwhich is taken by a camera module 204 of the image capturing device,which image is displayed in the viewfinder 206 of the device. The imagecapturing device further comprises the multi-microphone array 202,preferably arranged such that the audio capturing from the direction thecamera module is pointing at can be carried out most unobstructedly.

The directional multi-microphone array may comprise two or moremicrophones. Typically the minimum distance between the audio source(object) and any of the microphones in the array is greater than themaximum distance between the microphones. Therefore, the use of e.g. 3,4, 5 or more microphones may provide improved directional selectivity.

An example of functional units for implementing the embodiments isdescribed in FIG. 3. An image capturing unit 300 may comprise an imagesensor arranged to capture the plurality of images of the multi-shotseries according to the instructions received from a control unit 302.The control unit 302, in turn, gives the instructions in response to auser pressing the shutter button (not shown). In a similar manner, thecontrol unit 302 gives instructions, when the user presses the shutterbutton, to an audio capturing unit 304 to start the capture. The audiocapturing unit 304 may comprise said one or more microphones or themulti-microphone array. The plurality of captured images may betemporarily stored in a memory 306, or they may be subjected to animmediate selection process in a selection unit 308.

The selection process in the selection unit 308 is controlled by theinformation received from an audio analysis unit 310, wherein thecaptured audio signal associated with the plurality of images isanalysed in order to determine an auditory event relating to the desiredoutput image. As a result of this analysis, an image having mostprobably the best quality may then be selected in the selection unit 308as an output image 314 as such. Alternatively, one or more images fromthe multi-shot series may be selected in the selection unit 308 forfurther processing in a processing unit 312 in order to produce a betterquality output image 314. In the processing unit 312, various imageprocessing algorithms may be applied to the selected one or more imagesto derive a composed output image with enhanced visual quality, or if aplurality of images are selected for further processing, they may justbe combined to obtain an improved output image. The output image maythen be stored in storage means (not shown in FIG. 3) or shown on adisplay (not shown in FIG. 3), for example.

Various embodiments for implementing audio analysis unit 310 of FIG. 3,i.e. embodiments for analysing the captured audio signal associated withthe plurality of images in order to determine an auditory event relatingto the desired output image are described next more in detail.

According to a first embodiment, the analysis is carried out byestimating the direction of arrival of audio signal in order todetermine when the sound source is in desired part of the image.According to a further embodiment, a technique referred to asbeamforming can be utilised, wherein a multi-microphone array is used toestimate a direction of arrival of an audio signal.

The direction of arrival of the audio source, i.e. the location of theobject within the camera viewfinder, can be extracted using beamformingalgorithms. A generic microphone array consisting of a plurality (M) ofmicrophones in the beamformer configuration is presented in FIG. 4.

The output of the microphone array of FIG. 4 is determined as

$\begin{matrix}{{{y(n)} = {\sum\limits_{m = 1}^{M}{\sum\limits_{k = 0}^{L - 1}{{H_{m}(n)}{x_{m}\left( {n - k} \right)}}}}},} & (1)\end{matrix}$

where H (n) is the beamforming filter and L denotes the length of thebeamforming filters H₁, H₂, . . . , H_(M). The most trivial selection ofthe filters H₁, H₂, . . . , H_(M) are delay lines, thus giving theoutput of the microphone array as

$\begin{matrix}{{{y(n)} = {\sum\limits_{m = 1}^{M}{x_{m}\left( {n - \tau_{m}} \right)}}},} & (2)\end{matrix}$

where τ_(m) is the delay corresponding to the direction of arrival ofeach signal x₁, x₂, . . . , x_(M). The directionality may also beimplemented in the frequency sub-band domain or e.g. in the DFT(discrete fourier transform) transform domain. In that case the delayfor each microphone may be frequency-dependent.

The direction of arrival of sound source (see FIG. 2) compared tonominal axis perpendicular to the view finder image and microphone arrayis determined by searching the optimal delay value for each microphoneline. The delay τ_(m) is understood as the difference of the distancebetween the source and different microphones as follows

τ_(m)=(|s−x _(m) |−|s−x _(i)|)/c,  (3)

where s is the desired tracked audio source location coordinates, x_(m)is the location of the microphone m, x_(i) is the location of “zerodelay” microphone i. c is the speed of sound. It should be noted that inthis embodiment the delay in microphone i is always set to zero. Thezero delay, i.e. the microphone i could be allocated to any of themicrophones in the array.

According to the embodiment, the multi microphone beamformer is appliedin detecting the audio source location e.g. by searching for the filtercoefficients that maximise the microphone array output energy. In themost trivial case the delay values for each microphone line isdetermined in such a manner that the output energy is maximised

$\begin{matrix}{\tau_{m,{opt}} = {\arg {\max\limits_{\tau_{m}}\left( {{y(n)}^{T}{y(n)}} \right)}}} & (4)\end{matrix}$

The direction of arrival of audio source could be determined e.g. in therange of −π/2≦φ≦π2, where zero angle is perpendicular to the microphonearray. Based on the optimal delay for each microphone line, and usingthe knowledge of microphone position (distance between microphones), thedesired direction of arrival could be determined solving from theequation

τ_(m)=(|x_(m) x _(i)|sin (φ))/c  (5)

In this approach, the direction of arrival is identical to everymicrophone and the distance of the sound source is consideredsignificantly larger corresponding to the distance between microphones.Similar determination of time delays could be done in both horizontaland vertical direction when the microphone array design is twodimensional.

FIG. 5 shows an example of sound direction of arrival estimation for thecaptured audio signal corresponding to the series of images in FIG. 1.The corresponding sound direction of arrival is determined for each timeinstant when an image is captured. The scale of time axis of FIG. 5corresponds to that of FIG. 1 as illustrated therein below, and thescale of the direction of arrival is the range of −π/≦φ≦π/2, wherebyzero angle resides approximately in the midpoint of the vertical axis.

As can be seen, the curve estimating the direction of arrival of theaudio seems to reach the zero angle point (i.e. the object residingsubstantially perpendicular to multi-microphone array) at the timeinstant of the third image. This most presumably indicates that thesound source is in the middle of the viewfinder at the time the thirdpicture was taken, and therefore the third image can be considered mostpromising candidate for optimal output image.

According to a second embodiment, the analysis of the captured audiosignal associated with the plurality of images is carried out bydetermining the sound level of the audio signals. This is a ratherstraightforward approach, wherein an abrupt change in the sound levelmay be used to reveal the desired moment to capture the image. Theimplementation does not require a directional microphone array, but onlya single microphone could be used in capturing the audio signal. Themicrophone signal level could be determined sample by sample for exampleusing the equation

e(n)=y(n)^(T) y(n)  (6)

wherefrom a sound level estimation versus the time spent on capturingthe plurality of images can be created.

FIG. 6 shows an example of sound level estimation for the captured audiosignal corresponding to the series of images in FIG. 1. Thecorresponding sound level is determined for each time instant an imageis captured. The scale of time axis of FIG. 6 corresponds to that ofFIG. 1 as illustrated below. As can be seen, at the time instant of thethird image the sound level curve appears to have a sudden change andreaches the maximum level at the time instant of the third image. Thesudden change may be used as an indication that the sound source is inthe middle of the viewfinder or at least some special event has happenedapproximately at the same time the third picture was taken.Alternatively, or additionally, the maximum sound level may be used asan indication that the sound source was in the middle of the viewfinderapproximately at the same time the third picture was taken.

According to a third embodiment, the analysis of the captured audiosignal associated with the plurality of images is carried out bydetermining a change in the fundamental frequency of the sound of theobject. As is generally known from the physics of the Doppler effect,objects getting closer to the listener (or microphone) have a frequencyshift towards higher frequencies compared to the objects getting furtheraway from the listener. Thus, by determining the time instant when achange from a higher fundamental frequency to a lower fundamentalfrequency occurs, it can also be revealed when the object emitting thesound is closest to the listener, i.e. the image capturing device inthis context.

The fundamental frequency of the sound of the object can be determinedas an inverse of the time delay corresponding to the maximal value ofautocorrelation of the dominant audio source. Hence, a simplecorrelation analysis of the dominant audio source is preferably carriedout, wherein the delay τ_(n) within predetermined range of candidatevalues d maximising the autocorrelation is determined as follows

τ_(n)=arg max_(d){Φ_(n)(k,d)}  (7)

where Φ_(n)(d,k) is normalised correlation

$\begin{matrix}{{\Phi_{n}\left( {d,k} \right)} = \frac{{s_{n}\left( {k - d_{1}} \right)}^{T}{s_{n}\left( {k - d_{2}} \right)}}{\sqrt{\left( {{s_{n}\left( {k - d_{1}} \right)}^{T}{s_{n}\left( {k - d_{1}} \right)}} \right)\left( {{s_{n}\left( {k - d_{2}} \right)}^{T}{s_{n}\left( {k - d_{2}} \right)}} \right)}}} & (8) \\{{where}\mspace{14mu} \begin{matrix}{d_{1} = {\max \left\{ {0,{- d}} \right\}}} \\{d_{2} = {\max \left\{ {0,d} \right\}}}\end{matrix}} & (9)\end{matrix}$

and k is the sampling time instant and s_(n) is the sound signal.

Thereafter, the fundamental frequency of the sound of the object can bedetermined as the inverse of the delay value.

FIG. 7 shows an example of the fundamental frequency estimation for thecaptured audio signal corresponding to the series of images in FIG. 1.The corresponding fundamental frequency is determined for each timeinstant an image is captured. Again, the scale of time axis of FIG. 7corresponds to that of FIG. 1 as illustrated therein below. As can beseen, during the first two images the fundamental frequency is constanton a higher frequency, indicating an object getting closer to themicrophone of the image capturing device. Then at the time instant ofthe third image, there is a linear change in the fundamental frequencyto a lower frequency level, and thereafter the fundamental frequencyremains on that constant lower frequency, indicating the object gettingfurther away from the microphone of the image capturing device. Thus,the change during the third image from a higher fundamental frequency toa lower fundamental frequency indicates that the sound source was thenclosest to the image capturing device and most presumably in the middleof the viewfinder or at least some special event has happened about thesame time the third picture was taken.

According to an embodiment, the fundamental frequency estimation may beused to select an image in which the object is getting closer to themicrophone of the image capturing device; i.e. an image corresponding toa higher fundamental frequency. The information may be utilised, forexample, to get a front image of the object.

According to an embodiment, if the object is relatively far from thecamera, the limitation of the speed of sound may need to be taken intoaccount. If the distance to the object is known to be e.g. 340 meters, adelay of approximately one second needs to be considered when capturingand associating the audio information with the plurality of capturedimages. Naturally the delay must be adjusted in accordance with theactual or estimated distance to the object, for example with 500 mdistance the delay is approximately 1.5 seconds and with 170 m distancethe delay is approximately 0.5 seconds.

According to an embodiment, the information indicating the distance tothe object could be extracted, for example, from the focal information.Alternatively, the distance information could be estimated by astereoscopic camera, wherein two imaging sensors are situated at certaindistance within the same device, thus producing two images showingslightly different views of the scene. This difference is then exploitedto estimate the distance to a certain object of the scene.

A skilled man appreciates that any of the embodiments described abovemay be implemented as a combination with one or more of the otherembodiments, unless there is explicitly or implicitly stated thatcertain embodiments are only alternatives to each other.

The camera module can be implemented in the image capturing device as anintegral part of the device, i.e. as an embedded structure, or thecamera module may be a separate module, which comprises the requiredimage and audio capturing and/or processing functionalities and which isattachable to various kinds of data processing devices.

The functionalities of the embodiments may be implemented in the imagecapturing device preferably as a computer program which, when executedin a central processing unit CPU or in a dedicated digital signalprocessor DSP, affects the device to implement procedures of theinvention. Functions of the computer program SW may be distributed toseveral separate program components communicating with one another. Thecomputer software may be stored into any memory means, such as the harddisk of a PC or a CD-ROM disc, from where it can be loaded into thememory of device. The computer software can also be loaded through anetwork, for instance using a TCP/IP protocol stack.

It is also possible to use hardware solutions or a combination ofhardware and software solutions to implement the inventive means.Accordingly, the above computer program product can be at least partlyimplemented as a hardware solution, for example as ASIC or FPGAcircuits, in a hardware module comprising connecting means forconnecting the module to an electronic device, or as one or moreintegrated circuits IC, the hardware module or the ICs further includingvarious means for performing said program code tasks, said means beingimplemented as hardware and/or software.

It is obvious that the present invention is not limited solely to theabove-presented embodiments, but it can be modified within the scope ofthe appended claims.

1.-38. (canceled)
 39. A method comprising: receiving a plurality ofimages corresponding to a time period covering an intended moment;receiving an audio signal associated with the plurality of images;analyzing the received audio signal in order to determine an auditoryevent associated with a desired output image; and selecting at least oneof the plurality of images on the basis of the analysis of the receivedaudio signal for further processing in order to obtain the desiredoutput image.
 40. The method according to claim 39, wherein receivingthe plurality of images comprises capturing the plurality of imagesduring the period covering the intended moment using an image capturingunit in response to a user of an image capturing device giving a commandto release a shutter at the intended moment; and receiving the audiosignal comprises capturing an audio signal associated with the pluralityof images using audio capturing means.
 41. The method according to claim39, further comprising: receiving a plurality of images corresponding toa first time instant before the intended moment.
 42. The methodaccording to claim 39, further comprising: receiving a plurality ofimages corresponding to a second time instant after the intended moment.43. The method according to claim 39, further comprising: estimating thedirection of arrival of the audio signal in order to determine when theobject producing the audio signal is in desired part of the image. 44.The method according to claim 39, further comprising: estimating energyof the received audio signal in order to determine an event of interestwithin the plurality of images.
 45. The method according to claim 44,wherein an abrupt change in the energy of the received audio signal isused to determine the desired moment of the event of interest.
 46. Themethod according to claim 39, further comprising: estimating afundamental frequency of the received audio signal in order to determineat least one image from the plurality of images in which the objectproducing the audio signal is closest to the image capturing deviceproviding the plurality of images.
 47. The method according to claim 46,further comprising: using the estimation of the fundamental frequency toselect an image in which the object producing the audio signal isapproaching the image capturing device providing the plurality ofimages.
 48. An apparatus comprising: at least one processor; and atleast one memory including computer program code for one or moreprograms, the at least one memory and the computer program codeconfigured to, with the at least one processor, cause the apparatus toperform at least the following, receive a plurality of imagescorresponding to a time period covering an intended moment; receive anaudio signal associated with the plurality of images; analyze thereceived audio signal in order to determine an auditory event associatedwith a desired output image; and select at least one of the plurality ofimages on the basis of the analysis of the received audio signal forfurther processing in order to obtain the desired output image.
 49. Theapparatus according to claim 48, wherein the apparatus is further causedto: receive a plurality of images corresponding to a first time instantbefore the intended moment.
 50. The apparatus according to claim 48,wherein the apparatus is further caused to: receive a plurality ofimages corresponding to a second time instant after the intended moment.51. The apparatus according to claim 48, wherein the apparatus isfurther caused to: estimate the direction of arrival of the audiosignal, captured by a microphone array, in order to determine when theobject producing the audio signal is in desired part of the image. 52.The apparatus according to claim 51, wherein the apparatus is furthercaused to: determine that the object is in desired part of the imagewhen the object producing the audio signal residing substantiallyperpendicular to the microphone array.
 53. The apparatus according toclaim 48, wherein the apparatus is further caused to: estimate energy ofthe received audio signal in order to determine an event of interestwithin the plurality of images.
 54. The apparatus according to claim 53,wherein the apparatus is further caused to: determine that the desiredmoment of the event of interest is when there is an abrupt change in theenergy of the received audio signal.
 55. The apparatus according toclaim 48, wherein the apparatus is further caused to: estimate afundamental frequency of the received audio signal in order to determineat least one image from the plurality of images in which the objectproducing the audio signal is closest to the image capturing deviceproviding the plurality of images.
 56. The apparatus according to claim55, wherein the apparatus is further caused to: use the estimation ofthe fundamental frequency to select an image in which the objectproducing the audio signal is approaching the image capturing deviceproviding the plurality of images.
 57. A computer program productcomprising a computer-readable medium bearing computer program codeembodied therein for use with a computer, the computer program codecomprising: code for receiving a plurality of images corresponding to atime period covering an intended moment; code for receiving an audiosignal associated with the plurality of images; code for analyzing thereceived audio signal in order to determine an auditory event associatedwith a desired output image; and code for selecting at least one of theplurality of images on the basis of the analysis of the received audiosignal for further processing in order to obtain the desired outputimage.
 58. The computer program product according to claim 57, furthercomprising: code for estimating the direction of arrival of the audiosignal, received from a microphone array consisting of two or moremicrophones, in order to determine when the object producing the audiosignal is in desired part of the image.